In the last 20 years, many commercial entities have emerged that have a presence on the Internet. For example, many merchants now sell their wares online. In some situations, some merchants have become very large, selling massive quantities of goods over the Internet as online purchasing becomes more popular. The systems employed by such merchants include vast server banks that run many applications. Such servers and systems may generate vast amounts of data that indicates the health of such systems. For example, applications executed on different hosts may generate logs that document how the applications respond to each request they receive. Also, logs are typically created by the servers themselves that document the operational health of the hardware running the various applications of the online merchants.
Current servers and applications operated by online entities generate log data and other monitoring data that is stored for future reference. In some cases, the amount of log and monitoring data produced daily can be huge. For example, for some online merchants, it is not unheard of for the volume of log and monitoring data generated each day to be measured in the terabytes. Also, such data is typically generated and stored separately for each host in a bank such as a server bank. As a consequence, it can be very difficult to access such data in order to diagnose the health of servers and applications running on them in real time.